The present invention is related to a system and method for minimizing the chance of losing sequential data during the process of storage.
A common function of computer systems is the storage of sequential data received in unpredictable quantities and at unpredictable times. Such data may include changes in values relayed from sensors monitoring an industrial process, audio data relayed in packets from across a network, or text data ancillary to a continuous stream of video data where the ancillary data is relayed during blanking intervals between lines, fields or frames.
Where such sequential data will be received at a very high rate, it is typically received or xe2x80x98capturedxe2x80x99 using hardware that includes small, high-speed buffers for temporary storage of the data before it is to be written to or xe2x80x98storedxe2x80x99 in a larger storage device. For the sake of economy, this larger storage device is usually either dynamic RAM or a disk drive, or other storage devices that can impose greater latencies in time to store data than the buffers within the capture hardware, regardless of the quantity or other characteristics of the data.
Such storage devices also often impose additional latencies that are the result of the organization of the storage locations within the storage device. Storage locations within RAM storage devices are typically organized into arrays (called pages) of rows and columns of memory cells. The act of storing a block of data that involves storage locations within more than one row or page typically incurs a greater latency than storing the same size block of data such that only the storage locations within one row or page is involved. Storage locations within disk storage devices are typically dispersed among multiple storage surfaces, each surface being organized into tracks and sectors within tracks. The act of storing a block of data that involves storage locations of more than one sector, track or storage surface typically incurs a greater latency than storing the same size block of data such that only the storage locations of one storage surface, track or sector is involved.
As a result of these latencies, a common problem is the inability of the storage device to store all of such sequential data received within a given period of time. Such situations can occur if the storage device is also being used to store much more than just the sequential data, alone, or if there are frequent recurring accesses to the storage device by devices in the computer system other than the capture hardware. In such situations, the storage device may simply not be available for access by the capture hardware to store all of the sequential data received within a given period. However, such storage problems can also occur if the sequential data is written by the capture hardware to the storage device in too many smaller storage operations. The latencies inherent in any storage operation, regardless of the quantity of data stored, may accumulate to the point that there is insufficient time for the storage device to store all of the sequential data received within a given period.
There are prior solutions that involve combining smaller pieces of sequential data into larger pieces so as to reduce the number of individual storage operations. Such prior art solutions have typically involved the use of counters, timing intervals, or the repeated occurrence of a predetermined event to control the combining of sequential data, as well as when to perform storage operations. However, these approaches do not take into account the ability of a storage device to respond to a request to store data at any given time.
A method, apparatus and system for storing data wherein pieces of data are selectively combined when a request to store data is pending, and at least two pieces of data have been received since the request was made, and those at least two pieces of data can be combined into at least one combined piece of data configured to fit within the organization of storage locations within the storage device.